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Abstract: This paper investigated students' achievement for learning American Sign Language (ASL), using two different 
methods. There were two groups of samples. The first experimental group (Group A) was the game-based learning for ASL, 
using Kinect. The second control learning group (Group B) was the traditional face-to-face learning method, generally used 
in sign language skill training for students with hearing impairments. This study was separated into two phases. In Phase I: 
3D trajectory matching measurement algorithm, the Euclidean distance algorithm was employed to present the similarity 
between teacher and student datasets. Then, Phase II: Effectiveness of the Game-Based Learning system, showed the 
proposed framework of the game-based learning for sign language. Moreover, knowledge of sign language together with 
the corresponding actions were captured from three sign language experts using the knowledge engineering method. 
Then, the proposed game-based system would be analysed to provide students immediate feedbacks and suggestions 
based on the knowledge transfer from ASL experts. In the experiment, the students (N=31) were divided into two groups. 
The first group, Group A (N=17), learnt with the game-based learning while the second group, Group B (N=14), learnt with 
the traditional face to face learning method. The study result showed a significant difference (p<0.05) on the mean score of 
the post-tests for both Group A and Group B. It also presented that the game-based learning approach provided a better 
performance of ASL vocabularies than the traditional face-to-face learning approach. Finally, in the section of Discussion 
and Conclusion, the effectiveness and the future work opportunity of the proposed game-based learning system were 
discussed for the improvement of sign language actions. 

Keywords: Sign Language; American Sign Language; Recognition System; Kinect; Expert System; Game-Based Learning; 
Knowledge Engineering; 


1. Introduction 

Nowadays, schools for the deaf in Thailand commonly use the traditional teaching method to provide 
education for deaf people or students with hearing-impairments. These deaf schools use just black and white 
boards as a teaching tool to transfer knowledge. All communication learning is taught with an emphasis on 
learning development in Sign Language (SL) using lip-reading, writing, reading, listening, speaking and using SL 
to communicate with other people (Kamnardsiri et al., 2009). 

Sign Language (or Signed Language, SL) is a visual language which is used to communicate with hearing- 
impaired people. There are two types of sign language: (1) signs and (2) finger spellings (Yang, 2014). As a 
result, hearing-impaired people can communicate with normal hearing people using sign language. 
Nonetheless, not everyone uses the same sign language for communication. In order to communicate with 
foreign people with hearing impairments, a standard sign language is required, for instance, American Sign 
Language (ASL) is the sign language for communication with hearing-impaired Americans. However, there is a 
lack of experts to teach American Sign Language a lack of budget to support deaf education for hearing- 
impaired students. Other learning alternatives are considered as possible ways to help hearing-impaired 
students learn Sign Languages best. 

There have been developments on sensing technology which are employed in numerous kinds of research 
studies and also in sign language recognition including motion capture technology, data gloves, colour 
cameras, Microsoft's Kinect sensor, etc. (Ren et al., 2013). Many studies have been conducted with different 
diverse goals, for instance, gesture recognition (Schlomer et al, 2008; Xu et al., 2012; Sreekanth et al., 2017), 
hand gesture recognition (Cheng et al., 2013; Cheng et al., 2016; Maqueda et al., 2016), physical rehabilitation 
(Chang et al., 2011; Callejas-Cuervo et al, 2016; Spasojevic, 2017) and interactive displays (Morrison et al, 
2005; Zhang et al., 2012; Nathan et al, 2016; Wang et al, 2016) etc. In addition, computer games, especially 

ISSN 1479-4403 283 ©ACPIL 

Reference this paper as: Kamnardsiri T et al., "The Effectiveness of the Game-Based Learning System for the Improvement 
of American Sign Language using Kinect" The Electronic Journal of e-iearning Volume 15 Issue 4 2017, (pp283-296) 
available online at www.ejel.org 








The Electronic Journal of e-Learning Volume 15 Issue 4 2017 


serious and educational games, have provided significant successes for learning complex topics. Most Game- 
Based Learning (GBL) researchers emphasize on the effectiveness of learning such as motivation, learning 
achievements, strategies, engagement and behavioural patterns (Liang et al., 2010; Hamari et al., 2016; Hsieh 
et al., 2016; Tsai et al., 2016). There are numerous studies which are associated with effective Game-Based 
Learning; for example, Papastergiou investigated the learning effectiveness and motivational appeal of a 
computer game for the learning of Greek high school Computer Science (CS) curriculum (Papastergiou, 2009). 
Moreover, Cheng and Su developed a game-based learning system to improve self-efficacy for students' 
learning. They mapped the course content into the game to provide scenarios for learning (Cheng & Su, 2012). 
Furthermore, Lester and his colleagues designed a game-based learning environment named "Crystal Island: 
Uncharted Discovery" for upper elementary science education (Lester et al., 2014). Additionally, Hsiao and 
Chen developed a gesture interactive game-based learning (GIGL) approach for preschool children (Hsiao & 
Chen, 2016). Despite the essential aspect of game-based learning approach, in pedagogy or educational 
research, there are still some problems, namely, a lack of software and application tools to improve students' 
skills, especially in deaf schools, to learn Sign Languages. 

In this paper, we investigated the effectiveness of the Game-Based Learning system for the improvement of 
American Sign Language (ASL) skills in students with hearing impairments using Microsoft's Kinect sensor. 
Some Knowledge Management and Knowledge Engineering techniques were employed to capture knowledge 
from teachers (Schreiber et al., 1994). Moreover, the Euclidean Distance algorithm was used to compare the 
similarity of ASL 3D trajectory data between students and their teachers, and also to calculate the similarity 
score of each student for the feedback (Kamnardsiri et al., 2016a). 

2. Background and related works 

2.1 Sign Language 

Sign Language (SL) is one of the languages used to communicate with hearing-impaired people. An Oxford 
dictionary defines Sign Language as "a system of communication using visual gestures and signs, as used by 
deaf people" (Oxford, 2017), whereas a Cambridge dictionary defines Sign Language as "a system of hand and 
body movements representing words, used by and to people who cannot hear or talk" (Cambridge, 2017). Yang 
classifies Sign Languages into two types of actions: (1) signs and (2) finger spellings. Signs are hand motion 
sequence movement and hand configurations (Yang, 2014) and finger spellings are codes to represent letters 
and numbers of the alphabet using standardised finger positions (The Deaf Society, 2015). Sign languages are 
applied in various countries but they are not a universal language. There are many different Sign Languages in 
the world; for instance, British Sign Language (BSL), Thai Sign Language (TSL), Japanese Sign Language (or 
Nihon Shuwa, JSL), Spanish Sign Language (Lengua de signs o senas espanola, or LSE) and America Sign 
Language (ASL) (Wfdeaf, 2015). 

2.2 Sign Language Recognition 

There are many research studies that apply Sign Language recognition to solve problems such as image 
transformation (affine transformation, rotation, scaling and illumination) and noise and temporal 
segmentation (Jangyodsuk et al., 2014). Nai and his colleagues proposed a set of fast-computable depth 
features to classify static hand posture from a single depth image. The proposed features were extracted from 
pixels on randomly positioned line segments and evaluated with American Sign Language (ASL) finger spelling 
dataset with two new hand posture datasets. The classification accuracy of the approach was comparable to 
state-of-the-art methods.Their results also showed that training and testing speed was much faster compared 
to other methods (Nai et al., 2017). Lim and his colleagues proposed a feature covariance matrix based serial 
particle filter to isolate sign language recognition system. The fusion of the median and mode filters was used 
to extract the foreground and perform hand detection, then the region around the tracked hands was 
extracted to generate the feature covariance matrix of the tracked hand gesture. The results indicated that the 
hand trajectories as obtained through the proposed serial of hand tracking were closer to the ground truth at 
an 87.33 percent recognition rate for the ASL (Lim et al., 2016). Yang and his colleagues applied a hierarchical 
conditional random field (CRF) for recognising hand movement and a Boost Map embedding method to verify 
the hand shapes in three-dimensional space with the Kinect sensor. The results showed that this method 
achieved the recognition rate of 90.4 percent from signed sentence data (Yang et al., 2009; Yang & Lee, 2010; 
Yang, 2014). Jangyodsuk and his colleagues employed dynamic time warping (DTW) for sign trajectory 
similarity and the histogram of oriented gradient (HoG) features for hand shape representation to develop sign 
language recognition system. The results revealed that this method could achieve accuracy rate at 82 percent 
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in ranking signs in the ten matches (Jangyodsuk et al., 2014). Chai and his colleagues used 3D trajectory 
matching for sign language recognition system using Microsoft Kinect sensor to establish translation system. 
The system comprises two modes: (1) Translation Mode and (2) Communication Mode. This system focused on 
the communication between a general person and a hearing-impaired person that could be done via an avatar 
(Chai et al., 2013). 

2.3 Expert System 

Oxford dictionary defines an expert system (ES) as "A piece of software which uses databases of expert 
knowledge to offer advice or make decisions in such areas as medical diagnosis" (Oxford dictionary, 2017). In 
the field of computer science, an expert system is an artificial intelligent (Al). The basic idea of ES is to transfer 
human knowledge to a computer system. Knowledge information is then created inferences by many experts 
in the specific field to support users' problem-solving activities (consider, decide and forecast). Expert systems, 
for example, DENDRAL expert system, was created to analyse mass spectra (Feigenbaum & Buchanan, 1993), 
while MYCIN was built to diagnose infectious blood diseases and recommend antibiotics (Shortliffe, 1976). 
DIPMETER was developed for an Advisor Analysis of Data Gathered During Oil Exploration (Hunt, 1985). 
CADUCEUS was assembled for blood-borne infectious bacteria (Miller, 1984). Furthermore, R1 (later called 
XCON, for eXpert CONfigurer) expert system was established for an ordering processing (Kraft, 1984). An 
expert system generally consists of four components which are (1) users, (2) user interface, (3) knowledge 
base and (4) inference engine. Figure 1 shows the components of a general expert system; first of all, the users 
are humans who need to solve problems in a specific field using the expert system. Secondly, the user 
interface is a display that allows users to interact with the system such as entering problems into the system. 
Then, the system calculates and shows answers on the display based on the knowledge including facts and 
rules which are related to the problems. Finally, the inference engine provides a set of reasoning methods 
from experts, sometimes called axiom (Luconi et al., 1986). 



Figure 1: Components of a general expert system, modified from Luconi et al. (1986). 

2.4 Knowledge Engineering 

Knowledge engineering is the process of knowledge acquisition to generate knowledge-based systems from art 
into an engineering discipline. Studer and his colleagues defined three of Knowledge engineering frameworks 
which comprise: (1) CommonKADS, a further development of KADS (Schreiber et al., 1994) as the creation of a 
collection model that captures particular features of the knowledge-based systems to develop, (2) Model- 
based and Incremental Knowledge Engineering (MIKE) (Angele, 1996), a method for developing knowledge- 
based systems that include all steps of designing and implementing via specification from the initial elicitation 
and (3) PROTEGE-II (Tu et al., 1995) an approach aims to assist the knowledge-based systems development by 
reusing problem-solving method and ontologies (Studer et al., 1998). 

2.5 Game-Based Learning System 

Game-Based Learning (GBL) is one type of the games that focus on learning outcomes to combine an 
effectiveness of learning principles into game environments. The purpose of GBL is to improve self-confidence 
and problem-solving skills for learners (Liang, et al., 2010). Prensky and Prensky indicated that Game-Based 
Learning was " about fun and engagement and come together of serious learning as well as interactive 
entertainment into a newly-emerging and highly-exciting medium" (Prensky & Prensky, 2007). Liang and his 
colleagues had reviewed and analysed the GBL design process, they suggested eleven preliminary designs 
which comprises; (1) Constructing an exploratory learning environment, (2) Setting an explicit learning 
direction, (3) Screening the entire learning process, (4) Applying multimedia, (5) Placing educational 
challenges, (6) Applying cognitive apprenticeship, (7) Providing appropriate hints and feedback, (8) Ensuring 
accurate game-play mechanics, (9) Integrating community, (10) Giving learners an appropriate level of control 
and (11) Designing a user-friendly interface (Liang et al., 2010). Additionally, Garris and his colleagues designed 


www.ejel.org 


285 


ISSN 1479-4403 

















The Electronic Journal of e-Learning Volume 15 Issue 4 2017 


the Input-Process-Outcome (IPO) game-based learning model to improve a performance of children's learning 
as shown in Figure 2 (Garris et al., 2002). 

Some studies employed IPO to develop the system for learning, for example, Ghergulescu and Muntean 
proposed a Motivation Assessment-oriented Input-Process-Outcome (MotlPO) game model to design an 
educational game (Ghergulescu & Muntean, 2014). Yang and his colleagues, in addition, used the IPO model to 
improve the English learning of elementary school third graders (Yang et al., 2008). 



Input Process Outcome 

Figure 2: The Input, Process and Outcome (IPO) game-based Learning model, modified from Garris et al. 

( 2002 ). 

Furthermore, Liang and his colleagues created four extra designs into the game-based learning environment 
which were (1) to provide opportunities for system thinking, (2) to apply negative game-play mechanics to 
stimulate motivation, (3) to give learners an alternative experience of role playing and (4) to construct 
balanced game rules and to incorporate resource exchange mechanisms (Liang et al., 2010). Moreover, 
Prensky and his colleagues indicated that a game is a form of "organised ploy" (Prensky, 2001). Kim suggested 
that "gamification can add an extra level of motivation and incentive to many higher education activities" (Kim, 
2015). Heinich and his colleagues stated that a game is "an activity in which participants follow prescribed rules 
that differ from those of real life while striving to attain a challenging goal" (Heinich, 2002). Deterding and his 
colleagues addressed that games for education had a huge diversity of objectives for using and the GBL 
designed serious games which related learning objectives in a game universe with certain cognitive along with 
visual immersion and gameplay to gamification as "the use of game design elements in non-game contexts" 
(Deterding et al., 2011). Romero, furthermore, defined that gamification and also serious games pointed at 
supporting the learning objective through a positive learning as well as gaming experience using the game 
mechanics and rules such as competitive rules, a scoring system and learning through playing (Romero et al., 
2012). Likewise, Romero specified that a methodology for GBL consisted of six-phase methodology containing 
learning objectives, learning-centered need analysis, game modalities, game mechanics and rules, learning 
assessment and feedback and gaming and learning experience. This methodology was called HEXA-GBL 
(Romero, 2015). 



Figure 3: A methodology for Game-Based Learning (GBL) design and evaluation, modified from Romero (2015). 
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2.6 Kinect Characteristic 

According to the detail of Kinect, Smisek and his colleagues described that Kinect could be used to recognise 
both 3D points and images using a depth camera and an RGB colour camera. Kinect is a device for measuring 
which includes: IR camera and IR projector. IR camera is used to decode the IR projection pattern to 
triangulate 3D scene. The specification of IR image is 1280 x 1024 for 57 x 45 degrees FOV, 6.1 mm focal 
length, 5.2 jam pixel size. RGB camera, on the other hand, offers medium quality images with 1280 x 1024 pixel 
for 63 x 50 degrees FOV, 2.9 mm focal length and 2.8 |im pixel size. Depth image (d) of Kinect is the primary 
raw output of the depth in the scene that returns inverse depth (d). The depth resolution of Kinect is 0.5m - 
15m from a planar target and around 5° of the image centre (Smisek et al., 2013). 


IR Projector RGB camera IR camera 



Figure 4: Component of Kinect comprises Infra-red (IR) projector, IR camera and RGB camera (Kamnardsiri, 
2016a). 

2.7 3D Trajectory Matching 

Several researchers have been studying the processing of motion recognition and classification using trajectory 
matching for many years. The Euclidean distance is one of the metric functions for calculating a distance that 
conforms to the metric properties: non-negativity, identity, symmetry and triangle inequality. Moreover, the 
Euclidean distance is more competitive than other approaches, especially when dealing with a larger data size. 
The Dynamic Time Warping (DTW) algorithm (Rabiner & Juang, 1993) is efficient for one-to-one matching with 
a low computational cost. However, the Continue Dynamic Time Warping (CDTW) algorithm solved an 
accuracy problem better than both first algorithms (Cassisi et al., 2012). 

3. Methodology 

This study was separated into two phases. In Phase I, 3D trajectory matching measurement, a suitable 
Threshold (Ts) value was obtained for the comparison between training dataset (teachers) and testing dataset 
(students). Then, in Phase II, the effectiveness of the Game-Based Learning System was calculated to compare 
between the game-based learning group and the traditional face-to-face learning group. 

3.1 Phase I: 3D trajectory matching measurement algorithm 

In this phase, we focused on checking the similarity between America Sign Language (ASL) vocabularies of 
training and testing datasets with the appropriate threshold (Ts) value. The Euclidean distance algorithm was 
used for similarity checking. 

Data Collection 

Participants: Three teachers (expert at ASL) from Anusansunthorn School for the Deaf, Chiang Mai, Thailand 
and ten students from Chiang Mai University were participants in this study. 

Procedures: Data consisted of two sections (1) training data captured from three teachers with five ASL 
vocabularies (bear, butterfly, fish, goat and lion) and (2) testing data captured from ten students using the 
same set of the ASL vocabularies from the teachers. 


Results 


Training data: Five ASL vocabularies which were bear, butterfly, fish, goat and lion were collected from each 
teacher, using only the right hand. In order to check the similarity between training and testing datasets, the 
captured data from Kinect device were used to collect trajectory path of three teachers with five ASL 
vocabularies as shown in Figure 5. 
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Figure 5: Graph of 3D-Trajectory dataset captured from three teachers with five ASL vocabularies: (a) Bear, (b) 
Butterfly, (c) Fish, (d) Goat and (e) Lion. 

Testing data: The Euclidean distance algorithm was employed to measure the similarity between training and 
testing datasets. Euclidean distance algorithm was given as shown in Equation 1 (Cassisi et al., 2012). 


d(T,S) = 



( 1 ) 


where, k was time, d(T, S) was the distance between teachers (T) and students (S) datasets as given in 
Equation 2. 


d = J(Tx - Sx) 2 + (Ty - Sy) 2 + (Tz - Sz) 2 ' ' 

And also, the similarity between teacher and student datasets is presented in Equation 3. 

( 3 ) 

where, S (T, S) was the similarity value between teacher (T) and student (5), k was time, d(T, S) was the 
distance between teacher (T) and student ( S) and L was the length of data. 

In order to give flexibility to the measurement of ASL similarity between teacher and student datasets, 
threshold (Ts) values were set at 0.05 and 0.10. 


Table 1: The similarity results from ten students of five ASL vocabularies (Threshold =0.05). 


Signs 

Average Similarity of students compared with teacher (N=30)at Threshold ( Ts=0.05) 

Minimum 

Maximum 

Mean 

SD 

Variance 

Confidence Level (95.0%) 

Bear 

0.050 

0.637 

0.411 

0.186 

0.035 

0.341 

0.480 

Butterfly 

0.087 

0.562 

0.330 

0.114 

0.013 

0.287 

0.373 

Fish 

0.010 

0.462 

0.303 

0.121 

0.015 

0.257 

0.348 

Goat 

0.025 

0.562 

0.338 

0.138 

0.019 

0.286 

0.389 

Lion 

0.175 

0.525 

0.403 

0.079 

0.006 

0.373 

0.433 


Note: The maximum similarity value is 1.000. 


Table 2: The similarity results from ten students of five ASL vocabularies (Threshold =0.10). 


Signs 

Average Similarity of students compared with teacher (N=30) at Threshold ( Ts=0.10) 

Minimum 

Maximum 

Mean 

SD 

Variance 

Confidence Level (95.0%) 

Bear 

0.125 

0.825 

0.588 

0.159 

0.025 

0.528 

0.648 

Butterfly 

0.250 

0.725 

0.486 

0.119 

0.014 

0.442 

0.531 

Fish 

0.225 

0.575 

0.425 

0.071 

0.005 

0.398 

0.452 

Goat 

0.025 

0.787 

0.528 

0.175 

0.031 

0.462 

0.593 

Lion 

0.362 

0.625 

0.478 

0.051 

0.003 

0.459 

0.497 


Note: The maximum similarity value is 1.0. 
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Table 3: Average score results of testing five ASL vocabularies from ten students. 


Student 

Average score from three attempts (N=3) with Threshold ( Ts=0.10) 

Bear(10) 

Butterfly (10) 

Fish (10) 

Goat (10) 

Lion (10) 

Total (50) 

1 

3.33 

2.67 

0.00 

5.33 

0.00 

11.33 

2 

0.00 

0.00 

0.00 

4.00 

0.00 

4.00 

3 

1.33 

2.00 

0.00 

2.67 

0.00 

6.00 

4 

1.33 

1.33 

0.00 

2.67 

0.00 

5.33 

5 

2.67 

0.00 

0.00 

2.67 

0.00 

5.33 

6 

5.33 

1.33 

0.00 

0.00 

0.00 

6.67 

7 

3.33 

0.00 

0.00 

0.00 

0.00 

3.33 

8 

5.33 

0.00 

0.00 

0.00 

0.00 

5.33 

9 

6.00 

0.00 

0.00 

0.00 

0.00 

6.00 

10 

3.33 

2.67 

0.00 

3.33 

1.33 

10.67 
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Figure 6: Graph of Average score of testing with five ASL vocabularies comprises: Bear, Butterfly, Fish, Goat 
and Lion from ten students. 

The result of this phase consisted of two measurements which were the similarity testing and the score 
testing. For the similarity testing, we employed the Euclidean distance algorithm to evaluate between teacher 
and student datasets with the threshold value (Ts= 0.05 and Ts=0.10). Additionally, the score testing used IF- 
THEN rule for calculating the score and assess the student's test. The results are presented in Table 1, Table2, 
Table 3 and Figure 6. The results demonstrated that the sign with the most similarity between two datasets 
was Bear (Mean = 0.59, SD = 0.156, Variance = 0.025 and Confidence Level (95.0%) = 0.056) with Threshold (Ts) 
= 0.10. For the score testing, moreover, the highest average scores from the student dataset were 11.30 and 
10.67 out of the total score of 50. 


3.2 Phase II. The Effectiveness of the Game-Based Learning 

In this phase, we emphasised on the effectiveness of using the Game-Based system approach to compare with 
the traditional learning approach. The purpose of this phase was to collect pre-test, post-test data and then 
evaluate ASL. There were training and testing sections. To do this, ASL vocabularies were collected from 
teachers for training. Also students were tested with the same set of ASL vocabularies using the Game-Based 
System. 


Figure 7 showed the Framework that we designed for the Intelligent Game-Based System for Learning Sign 
Language. The system comprised of (1) User Interaction, (2) Graphic User Interface (GUI), (3) Expert System (4) 
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Trajectory Dataset and (5) Domain Experts. The new game-based system was developed to support both 
normal hearing people and hearing-impaired people in learning American Sign Language (ASL). The system was 
able to show the percentage of the correct movement actions as well as that of the incorrect actions and also 
the action scores. Motor-skills (ASL vocabulary) of users could be improved by themselves. 


Game-Based Learning System and Environment: The Game-Based System was developed by the Unity 3D- 
Game Engine and Kinect Xbox 360 from Microsoft Corporation. The Encode JSon format (Peng et al., 2011) was 
employed to create the data position (X, Y, Z) from both teachers and students. The configuration of the 
system environment was designed and used in this Phase as shown in Figure 8. 
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Figure 7: Framework of the Intelligent Game-Based System for Learning Sign Language modified from 
(Kamnardsiri et al., 2016b). 


Labtop/PC 


L 


Kinect 


0.80 m. 


2.00 m. 



Figure 8: Configuration of the Game-Based System Environment: Distance between Kinect Xbox 360 and 
standing point (Kamnardsiri et al., 2016). 

Procedures 

Participants: In this Phase, three teachers (ASL experts) from Anusansunthorn School for the Deaf, Chiang Mai, 
Thailand, along with 31 students from the Animation Department, College of Arts, Media and Technology, 
Chiang Mai University were participants. 

Data collection: The experiment was conducted in April 2016. Data comprised of: training data and testing 
data. Training data were gathered from three teachers with 20 ASL vocabularies: animal, alligator, peacock, 
zebra, giraffe, cat, tiger, snake, dog, elephant, dolphin, deer, penguin, cow, lion, panda, bear, butterfly, fish 
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and goat. Testing data were collected from 31 students with 5 ASL vocabularies including bear, butterfly, fish, 
goat and lion. Furthermore, the students were separated into two groups: (1) Group A using the game-based 
learning (N=17) and (2) Group B using the traditional face-to-face learning (N=14). Firstly, all students were 
given a 10-minute pre-test. Secondly, a 30-minute experiment was conducted by letting students in Group A 
use the game-based learning to practise ASL vocabularies skills. At the same time, students in Group B were 
taught by an instructor, using the traditional face-to-face learning. Finally, a 10-minute post-test was given to 
all students. The procedures of the experiment are shown in Figure 9. 


Group A (Experimental) 
(17 students) 




Group B (Control group) 
(14 students) 

i 




i 


Pre-test 



(ASL skill test) 

10 min. 

- 1 - 




i 

learned with 




learned with 

Game-based system approach 


the traditional approach 

30 min. 



30 min. 

T 




i 


Post-test 



(ASL skill test) 

10 min. 


Figure 9: The experimental procedures comprise: Group A: Experimental group (17 students) and 
Group B: Control group (14 students). 

Data analysis 

Training data: In this Phase, three sequences of 20 ASL vocabularies were collected from three teachers. Four 
points (the right hand, the left hand, the right shoulder and the left shoulder) were used to check the similarity 
between training and testing datasets. 

Testing data: The Euclidean distance algorithm was used to measure the similarity between training and 
testing datasets (Cassisi et al., 2012). In order to give flexibility to the measurement of ASL similarity between 
teacher and student datasets, a threshold (Ts) value was set at 0.10. 

Statistical Measurement 

To measure potential initial differences of five ASL vocabularies between Group A and Group B, the IBM SPSS 
17 Statistics software was used for analysis. For the first step, descriptive statistics such as mean, standard 
deviation (SD) and paired differences of five ASL vocabularies were analysed. The pre-test with Group A and 
Group B and the post-test with Group A and Group B were evaluated by using Univariate Analysis of Variance 
(ANOVA). Secondly, inference statistics for the measurement of the different teaching strategies with the 2x2 
between-groups analysis of covariance (ANCOVA) was conducted to calculate the effectiveness of the 
students' skills. The significance level was set at 0.05. 

Results 

Demographic Characteristics: Table 4 presents the demographic characteristics of student samples. The age of 
the majority of the students (54.8 percent) was 21 years old. The table also shows the ratio of the student 
genders which were male: female (52:48) and also the ratio of student groups which were the Game-based 
system for learning: the Control group learning (55:45). 
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Table 4: Demographic profiles of students. 


Demographic Profiles (N=31) 

Frequency 

Percent 

Age 

19 

1 

3.2 


20 

6 

19.4 


21 

17 

54.8 


22 

7 

22.6 

Gender 

Male 

16 

51.6 


Female 

15 

48.4 

Group of learning 

Group A (Game-based system for learning) 

17 

54.8 


Group B (Control group learning) 

14 

45.2 


Descriptive statistics: The mean scores of the learning performance test and ANOVA test which were given to 
students in both groups (Group A and Group B) as Pre-test and Post-test. 

The results included Bear, Lion, Goat, Butterfly and Fish signs. For the Pre-test, the students in Group A (the 
game-based learning) and the students in Group B (the traditional face-to-face learning) did not provide any 
significant difference in the mean score. For the Post-test, however, the average results of Group A were 
higher than those of Group B. According to the ANOVA test results of Group A and Group B, there was a 
significant difference (p < 0.05) as shown in Figure 10, for example, for the "Lion" sign: the Pre-test of Group A 
(N=17, Mean=20.38, SD=14.11), and of Group B (N=14, Mean=16.74, SD=10.87). The different results between 
both groups did not show a significant difference (F=0.626, Sig=0.435, R2=0.021). On the contrary, the Post¬ 
test of Group A (N=17, Mean=83.71, SD=4.25), and of Group B (N=14, Mean=78.70, SD=7.99), the results 
presenting the difference between both groups displayed a significant difference (F=4.992, Sig=0.033<0.05, 
R2=0.147) as shown in Table 5. 


Inference Statistical Measurement: In order to measure the achievement of learning ASL, when using the 
game-based learning compared to the traditional face-to-face learning, the ANCOVA was used to assess the 
effectiveness of the student skills via Pre-test and Post-test scores. As shown in the results, the comparison of 
the mean differences between the Pre-test taken by Group A and Group B demonstrated that the means were 
not different. This meant that there was no significant difference in the scores for all ASL vocabularies. 
However, the comparison of the average difference of the Post-test taken by Group A and Group B showed 
that there was a difference of mean or a significant difference in the scores with the significant level at 5% (P- 
value < 0.05) as shown in Table 6. 
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Figure 10: The comparison of Pre-test and Post-test of five vocabularies between Group A (Experimental 
group) and Group B (Control group). 
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Table 5: Descriptive statistics and ANOVA test for post-test of five ASL vocabularies. 


ASL Sign 

Post-test 

Group A (N=17) 

Group B (N=14) 

F 

Sig- 

R 2 

Mean 

SD 

Mean 

SD 

Bear 

73.61 

6.98 

56.15 

13.56 

21.41 

0.000 

0.42 

Lion 

83.71 

4.25 

78.70 

7.99 

4.992 

0.033 

0.15 

Goat 

83.06 

7.27 

57.69 

22.79 

18.86 

0.000 

0.39 

Butterfly 

80.11 

7.20 

58.37 

15.34 

27.07 

0.000 

0.48 

Fish 

81.92 

6.25 

66.24 

10.75 

25.72 

0.000 

0.47 


Note: Learning methods: Students in Group A used the game-based system learning while students in Group B 
used the traditional face-to-face learning. Level of significance was set at 0.05. 


Table 6: The comparison of the mean differences between Group A (the game-based system learning) and 
Group B (the traditional face-to-face learning) of Post-test from testing five ASL vocabularies (N=31). 


ASL Sign 

The comparison of the mean different of Post-test between Group A and Group B (N=31) 

F 

Mean 

Difference 

Std. 

Error 

Sig- 

(P-value) 

R2 Squared 

Confidence Interval (95%) 

Comparison 

Lower Bound 

Upper Bound 

Bear 

22.11 

17.97* 

3.82 

0.000 

0.44 

10.14 

25.78 

A>B 

Lion 

4.82 

5.06* 

2.30 

0.037 

0.15 

0.34 

9.78 

A>B 

Goat 

16.98 

24.78* 

6.01 

0.000 

0.40 

12.46 

37.12 

A>B 

Butterfly 

26.80 

21.99* 

4.24 

0.000 

0.498 

13.29 

30.69 

A>B 

Fish 

28.20 

16.35* 

3.07 

0.000 

0.50 

10.04 

22.65 

A>B 


Note: Learning methods: Students in Group A used the game-based system for learning while students in Group 
B used the traditional face-to-face learning. The sign * indicated significant level at 5% (P-value < 0.05). 


4. Conclusions and Discussion 

The study presented in this paper explored the effectiveness of the usability of the intelligent game-based 
system for learning sign language. First of all, for flow experience, it was concluded that: Phase I was about 
matching captured data from training (teacher) and testing (student) datasets, using the 3D trajectory 
matching measurement algorithm. This phase employed the Euclidean distance algorithm to measure the 
similarity between two datasets. In addition, Microsoft's Kinect Xbox 360 device was used to collect 3D 
trajectory data with the threshold value (Ts=0.0S and Ts= 0.10). The data of both teachers and students were 
encoded to create the data position (X, Y, Z) using JSon format (Peng et al., 2011). The results in Phase I 
showed that the value suitable for checking the similarity between teacher and student data was Ts=0.10. 
Secondly, in Phase II, the intelligent game-based system for learning sign language was developed by the Unity 
3D-Game Engine with Microsoft's Kinect device. Afterwards, the effectiveness of the game-based learning 
system was investigated to compare the learning performance between two experimental groups. Group A (17 
students) learnt SL with the game-based system. Group B (14 students) learnt SL with the traditional face-to- 
face learning. The experimental results of Phase II demonstrated that the performance of Group A students 
was higher than that of Group B students as shown in Figure 10 and Table 5. There was a significant difference 
in the scores with (P-value < 0.05) as shown in Table 6. 

According to the finding in the study, the benefits and limitations of using the intelligent game-based system 
to learn sign language are further discussed as follows. Firstly, the benefits of the system are the engagement 
of students to concentrate on ASL vocabularies and the enjoyment of the learning via the game. When the 
students attempted to gain victory on each vocabulary by achieving a higher score than the other group, the 
result was that the students had to use the motor skills to practise their movement. Furthermore, the system 
could offer a suggestion for the correct movement of each vocabulary. The system, moreover, was user- 
friendly in a sense that it was easy to use, configure and setup. In terms of the pedagogy, the proposed 
framework of intelligent game-based learning system was used to increase the performance of students to 
learn ASL vocabulary skills (actions or movements). Regarding the results from bar graph in Figure 10, it 
demonstrated that the scores of the post-test of each vocabulary were higher than the score of the pre-test. 
Hence, it implied that the system offered effectiveness for improving sign language skills of students. In 
addition, the students had the sense of control and concentration while playing the game, along with a good 
learning achievement as well. 
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Nevertheless, there were some limitations for this study: (1) The time limitation for the whole process of the 
proposed system, only six months, resulted in the small number of vocabularies used in the study. In real life 
situation, the quantity of vocabularies in the school curriculum is so much greater. The vocabularies used in 
this study were insufficient to be taught in different levels of the school and the system, therefore, the number 
of ASL vocabularies for the system could be increased, (2) The cost-effectiveness was directly linked to the 
increased number of ASL vocabularies. The 3D trajectory data from the teachers (experts) had to be collected, 
along with the suggestion sentences to correct the mistakes of the students' movements. Consequently, the 
cost of production would be higher by direct variation of ASL vocabularies. 

Finally, for future works, the larger sample size is recommended due to the small sample size in this study. 
Also, the suggestions and feedbacks from experts have to be considered and added to the game-based system 
for sign language learning. In addition, the proposed framework can be applied to the studies in other fields, 
for instance, in the medical field, a game-based system was used in studies helping brain-injured patients 
practise their movements (Betker et al., 2007; Lange et al., 2011; Molinari et al., 2016; Putnamet et al., 2016). 
In the sports field, additionally, game-based systems were applied to improve the performance of athletes 
(Kamnardsiri et al., 2015a; Kamnardsiri et al., 2015b; Eltoukhy et al., 2016). Hence, there are several 
opportunities to continue the research and/or to conduct new studies in this area. 
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